Papers with Chain-of-Thought reasoning
EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents (2025.acl-long)
Copied to clipboard
Cheng Qian, Peixuan Han, Qinyu Luo, Bingxiang He, Xiusi Chen, Yuji Zhang, Hongyi Du, Jiarui Yao, Xiaocheng Yang, Denghui Zhang, Yunzhu Li, Heng Ji
| Challenge: | Existing language model agents excel in planning and reasoning, but lack creativity in unfamiliar environments. |
| Approach: | They propose a benchmark suite of room escape game environments to challenge agents with creative reasoning, unconventional tool use and iterative problem-solving to uncover implicit goals. |
| Outcome: | The proposed framework can perform with 40% fewer steps and hints and performs robustly across difficulty levels. |
TopViewRS: Vision-Language Models as Top-View Spatial Reasoners (2024.emnlp-main)
Copied to clipboard
| Challenge: | Top-view perspective is a typical way in which humans read and reason over different types of maps, but spatial reasoning capabilities of modern VLMs in this setup remain unattested and underexplored. |
| Approach: | They introduce a top-view spatial reasoning dataset and use it to evaluate VLMs across 4 perception and reasoning tasks with different levels of complexity. |
| Outcome: | The proposed model can understand and reason over spatial relations from the top view and can be controlled at different granularities of spatial reasoning. |
A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains (2024.acl-long)
Copied to clipboard
Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, Mor Geva
| Challenge: | Recent literature discusses automatic methods to evaluate reasoning to improve their correctness, but no fine-grained step-level datasets are available to enable thorough evaluation of such verification methods. |
| Approach: | They propose to benchmark automatic verifiers of complex Chain-of-Thought reasoning in open-domain question-answering settings using a dataset that includes comprehensive labels for relevance, attribution to evidence passages, and logical correctness of each reasoning step. |
| Outcome: | The proposed dataset shows that verifiers struggle at verifying reasoning chains, particularly verifying logical correctness and detecting contradictions. |
KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional Knowledge Graph Question Answering (KGQA) methods rely on semantic parsing to retrieve knowledge strictly necessary for answer generation. |
| Approach: | They propose a retrieval-filtering-summarization pipeline that enhances QA coverage by retrieving a broader subgraph likely to contain relevant information. |
| Outcome: | The proposed pipeline surpasses state-of-the-art solutions by about 7% in quality and exceeds GPT-4o (Tool) by 10-21%. |
Chain-of-Exemplar: Enhancing Distractor Generation for Multimodal Educational Question Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multiple choice questions focus on text inputs and lack visual information. |
| Approach: | They propose a framework to generate subject-specific educational questions with plausible distractors based on multimodal content. |
| Outcome: | The proposed framework improves question generation and distractor generation over existing methods across subjects and educational levels. |
Traffic Light or Light Traffic? Investigating Phrasal Semantics in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Phrases are fundamental linguistic units through which humans convey semantics. |
| Approach: | They assess the capacity of API-based large language models to comprehend phrase semantics . they use three human-annotated datasets to analyze their results . |
| Outcome: | The proposed model outperforms embedding-based methods in phrase semantic reasoning tasks . the proposed model does not show significant advantage over fine-tuned methods . |
MolRAG: Unlocking the Power of Large Language Models for Molecular Property Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Recent LLMs exhibit limited effectiveness on molecular property prediction task due to semantic gap between representations and natural language and lack of domain-specific knowledge. |
| Approach: | They propose a framework that integrates Chain-of-Thought reasoning for molecular property prediction. |
| Outcome: | The proposed framework outperforms pre-trained LLMs on four datasets and matches supervised methods. |
Are LLMs Good Zero-Shot Fallacy Classifiers? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing fallacy classifiers lack sufficient labeled data for training, limiting their out-of-distribution (OOD) generalization abilities. |
| Approach: | They propose to use Large Language Models (LLMs) for zero-shot fallacy classification. |
| Outcome: | The proposed schemes outperform existing classifiers in OOD inference scenarios and opendomain tasks. |
MRFD: Multi-Region Fusion Decoding with Self-Consistency for Mitigating Hallucinations in LVLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Vision-Language Models often produce hallucinations due to the limited ability to verify information in different regions of the image. |
| Approach: | a new decoding method improves factual grounding by modeling inter-region consistency . the method identifies salient regions using cross-attention and generates initial responses for each . |
| Outcome: | a training-free decoding method reduces hallucinations and improves response consistency . the proposed method generates initial responses for each region and weights reliability weights among responses . |
Shifting from Ranking to Set Selection for Retrieval Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches primarily rerank top-k passages based on individual relevance, often failing to meet the information needs of complex queries in multi-hop question answering. |
| Approach: | They propose a set-wise passage selection approach and introduce SetR which explicitly identifies the information requirements of a query through Chain-of-Thought reasoning. |
| Outcome: | The proposed approach outperforms both proprietary LLM-based rerankers and open-source baselines in terms of answer correctness and retrieval quality. |
GCIG: GraphRAG-based Cross-document Instruction Generation for Boosting LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models struggle in knowledge-intensive domains and complex reasoning tasks due to their limited coverage of single-document knowledge and repetitive content. |
| Approach: | They propose a GraphRAG-based cross-document instruction generation framework that generates diverse questions through task-aware prompts and context-sensitive retrieval. |
| Outcome: | The proposed framework outperforms existing methods on knowledge-intensive and multi-hop question-answering tasks. |
SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on multimodal faithfulness have focused on perceptual hallucinations, raising concerns about the validity of reasoning traces. |
| Approach: | They propose a diagnostic benchmark that enforces explicit visual comparison to assess faithfulness of reasoning traces. |
| Outcome: | The proposed framework improves visual routing and aligns reasoning with perception. |
FAQ: Mitigating Quantization Error via Regenerating Calibration Data with Family-Aware Quantization (2026.findings-acl)
Copied to clipboard
| Challenge: | representativeness and universality of calibration data remain a bottleneck in quantization accuracy. |
| Approach: | They propose a framework that leverages prior knowledge from LLMs to generate calibration samples . their framework reduces accuracy loss by up to 28.5% compared to baseline . |
| Outcome: | Experiments show that family-aware quantization reduces accuracy loss by up to 28.5% compared to baseline data. |
Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent research indicates that Large Reasoning Models suffer from a strategic bottleneck at reasoning path planning. |
| Approach: | They propose a framework that reformulates reasoning as a dynamic search for the optimal thinking strategy. |
| Outcome: | The proposed framework improves accuracy and computational cost while reducing generation length by over 22%. |
ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to chain-of-thought reasoning incur high inference latency due to long generation traces. |
| Approach: | They propose a confidence-gated cascaded verification framework that reduces the trade-off between generation and verification. |
| Outcome: | The proposed framework achieves 2.24 speedups while matching target-model accuracy. |
DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain (2026.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language models lack fine-grained classification, single-view imagery, and inaccurate metadata. |
| Approach: | They propose a hierarchical, multi-view benchmark to evaluate VLMs across three levels of cognitive complexity. |
| Outcome: | The proposed benchmark evaluates vision-language models across three levels of complexity . it systematically identifies five primary failure modes . the proposed benchmarks are available on https://github.com/meituan/DiningBench. |
LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing KV cache compression methods mitigate memory bottlenecks but struggle in long reasoning tasks. |
| Approach: | They propose a lagged eviction framework that prioritizes evicts based on tokens’ recurrence patterns to reduce KV cache by 50% and maintain comparable accuracy. |
| Outcome: | The proposed framework reduces KV cache by 50% 70% while maintaining comparable accuracy, outperforming existing KV baselines. |
I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing (2026.acl-long)
Copied to clipboard
Jinghan Yu, Junhao Xiao, Chenyu Zhu, Jiaming Li, Jia Li, HanMing Deng, Xirui Wang, Guoli Jia, Jianjun Li, Xiang Bai, Bowen Zhou, Zhiyuan Ma
| Challenge: | Existing text-guided image editing methods rely on end-to-end pixel-level inpainting paradigm . existing models lack such intermediate representations and Reasoning-then-action process . |
| Approach: | They propose a "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment. |
| Outcome: | The proposed paradigm outperforms existing methods in compositional editing tasks. |